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PREFACE 


The  accurate  guidance  of  Its  strategic  and  conventional  cruise 
missiles  is  a  matter  of  great  concern  to  the  Air  Force.  There  ere  at 
present  two  methods  of  improving  the  location  accuracy  of  the  vehicle 
beyond  that  provided  by  the  onboard  inertial  system.  The  first  in¬ 
volves  time-of-arrival  techniques  in  the  Global  Positioning  System 
(GPS).  Earlier  Rand  studies  of  the  performance  cost  and  vulnerabilities 
of  this  system  have  shown  that  a  "survivable"  GPS  system  would  cost 
several  billion  dollars  and  still  may  be  vulnerable  to  jamming  in  the 
terminal  area;  thus,  terminal  delivery  in  the  presence  of  jasming  may 
not  be  accurate  enough  to  allow  for  the  delivery  of  nonnuclear  munitions 

The  second  method,  a  potentially  cheaper  alternative  to  GPS  guid¬ 
ance,  is  correlation  guidance.  A  correlation  guidance  system  using 
terrain  contours  (TERCOM)  is  configured  as  the  heart  of  the  guidance 
system  for  the  present-generation  cruise  missile.  Eventually,  there 
will  be  a  need  for  a  navigation  system  that  can  go  anywhere  in  the 
world,  including  to  the  flat  areas  where  terrain-contour  navigation 
falls,  and  possibly  for  the  delivery  of  nonnuclear  munitions  on  both 
strategic  and  tactical  targets.  Correlation  guidance  schemes  using 
imagery  (instead  of  terrain  contours)  along  the  midcourse  flight  path 
and  in  the  terminal  area  are  a  potential  means  of  achieving  these  goals. 

Current  Rand  studies  are  providing  a  better  understanding  of  the 
basic  principles  and  limitations  of  the  image-correlation  system.  They 
should  also  provide  a  methodology  for  improving  the  scene  selection 
process  and  yielding  a  higher  accuracy  per  fix.  Aimed  at  the  problems 
encountered  in  using  imagery— especially  those  of  radiometrics,  in  which 
the  Air  Force  is  heavily  engaged — this  Note  is  Intended  to  be  a  first 
step  in  providing  a  unified  theory  for  describing  all  matching  processes 
(both  pattern  recognition  and  correlation)  and  for  understanding  the 
effects  of  inherent  scene  characteristics  on  the  performance  of  the 
system. 

This  work  was  performed  under  the  Project  AIR  FORCE  research  pro¬ 
ject  "Battle  Management  System  for  ICBMs,  Bombers,  and  Cruise  Missiles." 
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This  Kota  should  ba  of  general  interest  to  researchers  in  tha 
ft aids  of  aap  aatehing  and  pat tarn  recognition,  aa  wall  aa  to  policy 
planners  concerned  with  aap  aatehing  for  weapon  guidance. 

This  Note  la  tha  second  in  a  sarlaa  dealing  with  aap  aatehing. 

It  uses  tha  basic  structure  of  tha  first  Mote,  N-1216-AF,  as  a  starting 
point  for  discussing  perforaance  considerations  associated  with  aap 
aatehing  systems.  This  Mote  and  tha  previous  Mote  approach  the  aap 
aatehing  problaa  from  a  physical  point  of  view. 
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I.  INTRODUCTION 


As  discussed  in  N-1216-AF,  the  correlation  process  can  be  struc¬ 
tured  as  shown  in  Fig.  1.  This  structure  reduces  the  error  sources 
to  four  generic  categories,  describes  the  preprocessing  as  either 
intensity  or  spatially  related,  places  all  matching  algorithms  into 
four  categories  (ranging  from  pure  feature  matching  to  ordinary  cor¬ 
relation),  and  describes  scenes  statistically  as  homogeneous  or  heter¬ 
ogeneous,  with  each  homogeneous  region  described  by  the  number  of 
independent  elements  it  contains. 

The  major  advantage  of  structuring  the  matching  process  in  this 
manner  is  that  the  problem  segments  itself  nicely,  as  shown  in  Fig.  2, 
in  associating  error  accommodation  with  a  single  element  or  several 
separate  elements  of  the  system.  Global  errors  are  accommodated  in 
preprocessing.  Geometric  distortions  are  handled  by  spatially  grouping 
the  elements  so  that  the  system  performance  is  not  severely  degraded 
by  the  maximum  amount  of  distortion  anticipated.  Global  errors  of  an 
intensity  nature  are  accommodated  by  normalizing  the  data  in  preproces¬ 
sing  before  any  matching  is  attempted.  Nonstructural  errors  can  be 
accommodated  to  some  extent  in  mission  planning.  Routing  to  avoid  po¬ 
tential  obscuration  and  masking  problems  can  reduce  system  vulnerabil¬ 
ity  to  these  errors.  Timing  of  the  mission  can  also  have  an  impact  on 
shadows,  area  blockages,  and  possibly  weather-related  effects  over  the 
target  area.  There  will  always  be  some  nonstructured  errors  which  can¬ 
not  be  predicted  or  accommodated,  and  one  must  hope  that  their  effect 
on  system  performance  will  not  be  significant.  Regional  errors  can 
either  be  accommodated  by  improved  signature  modeling  or  by  designing 
the  matching  algorithm  to  be  invariant  to  regional  errors.  One  extreme 
is  to  tailor  the  signature  to  the  arrival  of  the  vehicle  over  the  tar¬ 
get  area,  and  the  other  is  to  do  little  modeling  of  the  signature  but 
compensate  for  it  by  using  a  feature  matching  algorithm.  Compensation 
of  local  errors  is  generally  achieved  only  in  the  matching  algorithm. 

It  is  the  purpose  of  this  Note  to  examine  the  effect  of  these 
various  errors  and  of  the  scene  composition  on  the  performance  of  map 


Performance 

•  Probability  of  comet 

•  Aoouracy 


1  —  Generic  overview  of  map  matching  process 
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Fig.  2  —  Error  accommodation 

matching  systems.  The  two  performance  indices  of  interest  are  the 
probability  of  correct  match,  Pc>  and  the  accuracy  associated  with 
the  fix  position.  They  can  be  considered  independently  of  each  other, 
as  indicated  in  Fig.  3.  The  correlation  process  can  usually  be  im¬ 
plemented  in  two  phases ;  the  first  is  an  acquisition  phase  (designed 
to  locate  the  general  vicinity  of  the  match  position)  and  the  second 
is  to  achieve  maximum  accuracy.  In  the  discussion  that  follows,  these 
topics  will  be  treated  separately. 

In  this  Note  we  assume  that  the  reference  map  is  larger  than  the 
sensed  image,  and  that  the  sensed  image  is  wholly  contained  within  the 
boundaries  of  the  reference  map.  The  reference  and  sensor  maps  can 
be  broken  down  statistically  into  a  set  of  features  or  homogeneous  re¬ 
gions  which  comprise  the  entire  map.  Each  region  can  be  considered 
to  be  made  up  of  a  number  of  independent  elements.  An  independent 
element  may  be  composed  of  one  or  more  sensor  elements  or  pixels  de¬ 
pending  upon  the  degree  -»f  correlation  between  elements.  Thus,  the 
hierarchy  of  scene  structure  becomes  scene,  region,  independent  ele¬ 
ment,  and  element. 

Section  II  of  this  Note  describes  the  various  factors  which  in¬ 
fluence  system  accuracy  and  cause  false  matches.  Section  III  describes 
techniques  for  improving  system  accuracy. 
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Section  IV  discusses  acquisition  problems  and  ie  broken  into  two 
segments.  The  first  segment  deals  with  the  scene  selection  problem 
relative  to  acquisition,  and  the  second  segment  examines  the  possible 
design  tradeoffs  between  error  types  and  matching  algorithms. 


II.  UNDERLYING  FACTORS  AFFECTING  SYSTEM  PERFORMANCE 


The  output  of  the  matching  process  will  either  be  a  correct  match 

it' 

(where  one  is  concerned  with  the  accuracy  of  the  fix)  or  a  false  fix 

(where  one  Is  concerned  with  measuring  the  probability  of  occurrence 

(Pr)  or  nonoccurrence  (P  )). 
r  c 

ACCURACY 

Intuitively,  one  can  qualitatively  relate  the  system  accuracy  to 
the  shape  of  the  correlation  surface  surrounding  the  time  match  loca¬ 
tion.  A  sharp  peak  at  this  point  would  be  Indicative  of  a  high  accu¬ 
racy  fix;  a  flattened  surface  surrounding  the  correct  match  position 
would  suggest  degraded  accuracy.  Three  questions  arise  about  system 
accuracy.  What  drives  it?  How  does  one  quantify  it?  How  does  one 
develop  methods  for  improving  it?  One  might  also  ask  how  the  param¬ 
eters  of  accuracy  and  probability  of  correct  match,  Pc,  relate  to  one 
another.  They  are  obviously  related  in  some  sense,  but  the  question 
is  only  academic.  If  one  considers  a  two-phase  implementation  process 
in  which  the  first  phase  is  designed  for  maximizing  acquisition  and  the 
second  phase  (possibly  with  different  algorithms  and  preprocessing 
from  the  first  phase)  is  designed  to  maximize  the  accuracy  of  the 
position  fix,  then  one  is  not  concerned  with  their  relationship  because 
they  can  be  considered  independently. 

The  discussion  of  accuracy  will  focus  on  examining  the  major  driv¬ 
ing  factors  in  the  process  and  a  means  of  quantifying  these  factors. 
This  section  will  also  suggest  methods  for  improving  accuracy;  however. 


Accuracy  is  not  taken  here  to  be  the  ultimate  system  accuracy 
which  involves  interpolation  of  the  correlation  peak,  but  rather  accu¬ 
racy  is  taken  to  refer  to  the  width  of  the  region  around  the  true  cor¬ 
relation  peak  in  which  a  correlation  point  is  likely  to  occur.  The 
width  of  this  region  can  be  measured  in  map  displacement  positions 
which  are  generally  one  pixel  apart.  In  general,  accuracies  have  been 
reported  by  participants  in  the  DARPA  Terminal  Homing  Program  to  be  on 
the  order  of  one-pixel  bias  error  with  an  additional  one-pixel  stan¬ 
dard  deviation  about  that  bias.  Thus,  the  absolute  accuracy  will 
depend  upon  the  ground  resolution  of  the  sensor. 


that  topic  will  be  considered  in  more  detail  in  the  following  section. 

It  will  be  shown  that  the  degree  of  scene  nonhomogeneity  and  geometric 
errors  are  the  major  parameters  that  influence  accuracy.  The  discussion 
which  follows  starts  with  the  homogeneous  random  map  case  and  after 
nonhomogeneities  are  introduced  into  the  scene  drastic  changes  are 

N. 

observed  in  the  correlation  surface.  An  examination  of  real-world 
scenes  shows  that  accuracy  can  be  improved  by  decomposing  the  scene 
into  homogeneous  segments.  This  procedure  is  examined  in  more  detail 
in  Section  III. 

Let  us  start  by  examining  how  scene  composition  affects  the  corre¬ 
lation  surface.  First  we  will  consider  a  completely  random  scene  with 
elements  generated  from  a  Gaussian  distribution  with  zero  mean  and  unit 
variance.  The  random  scene  consists  of  20  *  20  elements  (pixels)  which 
we  shall  call  the  reference  map.  From  the  center  of  this  map  we  shall 
extract  a  10  x  10  element  sensor  scene  which  we  shall  designate  the 
control  map.  This  sensor  scene  is  then  correlated  with  respect  to  the 
reference  map  to  obtain  the  horizontal  (x,y)  autocorrelation  contours 

(correlation  height  versus  displacement  position)  about  the  center  of 

* 

the  map  (see  Fig.  4(a)).  Also  displayed  in  Fig.  4(a)  is  an  estimate 
of  the  number  of  independent  elements  in  the  scene.  For  this  control 
map  with  no  interpixel  correlation  (because  all  scene  elements  are  inde¬ 
pendently  randomly  generated)  the  estimation  process  determined  there 
to  be  97  independent  elements  (N^)  in  a  scene  of  100  elements  (N).  We 
can  use  this  parameter  Nj  as  a  quantitative  measure  both  of  the  number 
of  independent  elements  in  the  scene  and  of  the  spreading  and  contrac¬ 
tion  of  the  correlation  peak  about  the  match  point. 

To  simulate  the  effect  of  nonhomogeneity  introduced  into  a  random 
scene,  the  control  scene  was  altered  by  adding  a  constant  value,  p,  to 
all  the  map  elements  in  the  left-hand  portion  of  the  scene,  as  illus¬ 
trated  in  Fig.  5.  The  mean  level  between  the  two  regions  was  varied 
such  that  the  bias  level  difference  between  the  two  regions  would  be 
comparable  to,  or  larger  than,  the  variation  in  the  mean  intensity  of 
each  region.  Four  bias  levels  were  chosen:  (1)  V  *  0  (control  case 
shown  in  Fig.  4(a)),  (2)  p  ■  1  (Fig.  4(b)),  (3)  p  ■  10  (Fig.  4(c)), 

ft  „ 

Taken  from  R-2211-AF,  Estimation  Techniques  and  Other  Work  on 
Image  Correlation ,  J.  A.  Ratkovic  et  al.,  September  1977. 


(Ion  1 


Region  2 


Fig.  >  —  Geometry  tor  ohMge  in  Mot  Issete  bttwssn  regions 


and  (4)  y  •  100  (Fig.  4(d)).  Aa  scar.  In  Fig.  4,  aa  a  bias  level  be¬ 
tween  regions  la  added  (with  the  magnitude  of  the  bias  level  comparable 
to  the  variation  in  signal  level  in  each  homogeneous  region),  the  cor¬ 
relation  contours  become  more  drawn  out  and  the  estimate  of  the  number 
of  Independent  elements,  Nj,  decreases  to  a  value  between  the  maximum 
number  governed  by  the  total  number  of  independent  scene  elements  (100) 
and  the  minimum  number  governed  by  the  number  of  homogeneous  regions 
contained  within  the  scene  (2).  As  the  bias  level  between  regions 
becomes  significantly  greater  than  the  intensity  variation  in  each 
region  (y  ■  10  and  y  ■  100  cases),  the  correlation  contours  become 
flatter  and  flatter,  and  the  estimated  number  of  independent  scene  ele¬ 
ments  becomes  smaller  (Nj  reduces  to  two  as  y  »  c  intensity) . 
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The  conclusion  to  be  drawn  fro*  this  staple  example  Is  that  for 
nonhoaogeneous  scenes  (practically  all  real-world  scenes),  when  the 
variation  in  intensity  within  a  region  is  saall  relative  to  the  aean 
intensity  level  differences  between  regions,  the  correlation  process 
is  dominated  by  the  number  and  size  of  homogeneous  regions  composing 
the  scene.  It  would  then  appear  that  in  most  real-world  situations 
where  the  map  covers  a  number  of  homogeneous  regions  the  intensity 
level  variation  within  a  homogeneous  region  has  very  little  effect  on 
the  correlation  surface.  To  demonstrate  this  hypothesis,  we  selected 
several  areas  (agricultural,  mountain,  suburban,  and  desert)  from  an 
Earth  Resources  Satellite  (ERTS)  map  to  conduct  an  experiment.  In 
this  experiment  we  divided  the  maps  (both  sensor  and  reference)  into 
homogeneous  regions  and  replaced  the  Intensity  value  of  each  pixel  in 
a  region  with  the  average  intensity  level  of  the  region.  The  results 
of  this  experiment  are  shown  in  Fig.  6.  It  is  seen  that  the  correlation 
contours  with  and  without  intensity  variations  in  the  region  are  very 
similar,  indicating  that  the  mean  intensity  value  of  the  homogeneous 
region  is  the  dominant  factor  in  the  correlation  process  and  not  the 
individual  interpixel  variations  within  the  regions,  assuming  the  in¬ 
tensity  variation  within  the  region  to  be  small  relative  to  the  differ¬ 
ence  in  mean  intensity  level  between  homogeneous  regions. 

Thus,  from  the  aforementioned  experiments  it  appears  that  the  num¬ 
ber  of  homogeneous  regions,  their  size,  and  the  mean  intensity  value 
associated  with  the  region  are  the  major  scene  contributors  to  the 
correlation  process.  From  a  descriptive  point  of  view  we  can  consider 
a  random  (zero  correlation  between  pixels)  homogeneous  scene  in  one 
dimension  (as  illustrated  in  Fig.  7)  correlating  to  a  first  approxima-  I 

tion  as  a  Dirac  delta  function.  On  the  other  hand,  nonhoaogeneous 
scenes  can  be  viewed  as  a  first  approximation  in  one  dimension,  as  a 
series  of  square  waves  with  small  amplitude  variations  about  the  mean 
amplitude  level  due  to  Interpixel  Intensity  differences  within  the  re¬ 
gion.  The  autocorrelation  surface  for  this  process  then  looks  like  the 
convolution  of  two  square  waves  with  the  width  of  the  peak  around  the 
match  point  being  determined  by  the  size  of  the  homogeneous  region. 


Agricultural  Area  I  Desert  Area 
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Non homogeneous  scene 
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In  general,  there  Is  also  structure  within  a  homogeneous  region 
which  can  be  categorized  by  "independent  elements"  within  the  region. 

It  is  this  finer  structure  which  allows  correlation  systems  to  work  in 
homogeneous  regions;  otherwise,  if  there  were  only  one  independent 
element  within  the  region  (i.e.,  constant  Intensity  level),  one  could 
not  distinguish  a  correlation  peak.  As  a  second  approximation,  then, 
one  could  view  the  intensity  variations  contained  within  the  homogeneous 
regions  shown  in  Fig.  8  as  independent  elements.  Each  region  can  be 
considered  composed  of  a  number  of  Independent  elements  whose  size 
ultimately  limits  the  resolution  of  the  scene.  Each  Independent  ele¬ 
ment  in  turn  is  composed  of  a  number  (at  least  one)  of  spatially  con¬ 
nected  pixels  which  can  be  considered  to  be  independent  of  neighboring 
groups  of  independent  pixels.  Thus,  an  overview  of  the  process  might 
consider  the  signal  to  be  modulated  by  the  size  and  magnitude  of  the 
homogeneous  regions  (low  frequency  modulation)  and  the  size  and  mag¬ 
nitude  of  the  independent  elements  contained  within  the  region  (high 
frequency  modulation) . 

Up  to  this  point  we  have  shown  that  the  correlation  peak  is  gen- 

* 

erally  dominated  by  the  nonhomogeneous  characteristics  of  the  scene. 
There  is  also,  however,  some  contribution  to  the  peak  width  from  inter¬ 
pixel  correlation,  i.e.,  if  we  were  to  remove  the  nonhomogeneity  from 
the  scene  we  would  still  obtain  some  spreading  of  the  correlation  sur¬ 
face  due  to  independent  elements  contained  within  the  region.  This  is 
where  the  literature  concerning  the  subject  becomes  somewhat  vague. 
Generally  the  literature  uses  the  term  "correlation  length”  to  describe 
the  width  of  the  correlation  surface  and  uses  this  term  for  both  the 
interpixel  correlation  between  elements  and  the  total  scene  correlation. 
There  are  in  fact  two  correlation  terms — the  interpixel  correlation 
which  we  shall  define  by  a  correlation  length,  a,  and  the  total  scene 
correlation  which  we  shall  define  as  the  scene  resolution  parameter, 

N/Nj.,  where  N  is  the  total  number  of  resolution  elements  (pixels)  in 
the  scene.  These  terms  cannot  be  equated  unless  the  scene  is  homogeneous 

Mean  intensity  levels  of  regions  within  the  scene. 


Homogeneous ' 
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Signal  Intensity  level 
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Fif.  8  —  Seam  decomposition  into  regions  and  rndnpandtnt  ttamonts 
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To  Illustrate  these  points  ve  will  decompose  the  scene  into  homo¬ 
geneous  regions  and  autocorrelate  a  sensor  and  reference  scene  using 
the  standard  product  algorithm,  a  feature  matching  algorithm,  and  a 
hybrid  correlation  matching  algorithm,  all  of  which  were  described  in 
N-1216-AF.  The  feature  matching  algorithm  essentially  removes  the  ef¬ 
fect  of  homogeneous  regions  since  all  homogeneous  regions  are  zero- 
meaned  and  normalized  separately.  The  hybrid  algorithm,  on  the  other 
hand,  takes  out  some  but  not  all  of  the  effects  of  the  scene  nonhomo¬ 
geneity.  Figure  9  shows  the  effect  of  using  these  three  different 
algorithms  upon  the  correlation  surface  for  four  different  scenes. 

The  normal  autocorrelation  process  produces  a  spread  out  correlation 
peak,  while  the  feature  matching  algorithm  (homogenizing  both  the 
reference  and  sensor  scene)  produces  the  sharpest  correlation  peak, 
being  limited  only  by  the  interpixel  correlation.  The  hybrid  algorithm 
produces  a  correlation  surface  between  the  two  indicating  that  it  does 
remove  some  but  not  all  of  the  effects  of  scene  nonhomogeneity. 

Many  authors  in  the  field  have  shown  that  accuracy  can  be  improved 
through  the  use  of  edge  operators  (gradients,  Laplacians,  etc.)  or  the 
use  of  spatial  frequency  filtering.  These  results  are  consistent  with 
the  explanation  given  here  for  the  effect  of  homogeneous  regions  on 
system  accuracy.  Spatial  frequency  filters  which  cut  off  the  lower 
spatial  frequencies  associated  with  homogeneous  regions  will  improve 
system  accuracy  (provided,  of  course,  that  acquisition  has  been  ac¬ 
complished).  Similarly,  edge  operators  will  significantly  reduce  the 
broadening  effects  of  homogeneous  regions  and  their  effectiveness 
should  be  equivalent  to  correlation  matching  techniques  which  account 
for  features  in  the  scene  (feature  matching  correlation  and  hybrid 
correlation  algorithms). 

To  summarize,  the  dominant  scene  parameters  contributing  to  the 
correlation  process  are  the  size  and  number  of  homogeneous  regions 
composing  the  scene.  The  interpixel  correlations  and  intensity  varia¬ 
tions  between  pixels,  represented  by  the  number  and  size  of  independent 
elements  within  the  region,  are  only  significant  for  completely  homo¬ 
geneous  scenes  (which  are  rare)  and  for  scenes  which  have  been  homo¬ 
geneously  processed.  Conversely,  by  homogeneously  segmenting  the  scene, 


Agricultural  Area 


Normal  Autocorrelation 


sharper  correlation  peaks  can  be  produced  whose  widths  are  limited  only 
by  the  interpixel  correlation  or  the  size  of  the  independent  elements. 

The  effects  of  geometric  distortions  on  the  correlation  surface 
have  been  described  in  the  literature  and  are  similar  to  the  effects 
of  introducing  nonhomogeneity  into  the  scene.  Basically,  geometric 
distortions  spread  out  the  correlation  peak  and  reduce  its  maximum 
value.  For  illustrative  purposes,  Table  1  shows  how  the  degree  of 
congruence  between  the  sensor  and  the  reference  image  is  decreased  as 
the  scale  factor  (p)  is  increased. 

The  effect  of  regional,  local,  and  nonstructured  error  sources  is 
to  reduce  the  peak  of  the  correlation  surface  at  the  match  point,  but 
not  necessarily  to  spread  out  the  correlation  surface. 

OCCURRENCE  OF  FALSE  MATCHES 

There  are  two  basic  mechanisms  (in  the  extreme)  which  can  cause 
false  peaks  to  dominate  the  correct  match  peak.  These  basic  mechanisms 
can  be  attributed  to  either  random  noise  or  repetitive  spatial  patterns 
in  the  scene.  For  the  latter  mechanism,  we  consider  the  situation 
where  the  spatial  pattern  of  homogeneous  regions  within  the  scene  is 
replicated  or  nearly  replicated  in  other  parts  of  the  reference  scene. 
Extreme  examples  where  these  individual  mechanisms  can  cause  a  false 
match  by  themselves  can  be  illustrated  by  a  random  map  case  and  by  a 
checkerboard  pattern. 

In  the  case  of  a  "randomly  generated"  map,  there  is  not  (or  at 
least  should  not  be)  any  spatial  structure  to  the  map  so  that  the 
addition  of  a  sufficient  quantity  of  noise  can  result  in  a  false  match. 
The  location  of  the  false  match  position  should  occur  randomly  through¬ 
out  the  correlation  surface  with  different  noise  samples.  In  the  other 
case,  the  existence  of  a  repetitive  pattern  for  the  reference  scene 
(e.g.,  multiple  smokestacks)  can  lead  to  a  false  match  even  in  the 
absence  of  any  significant  system  errors  such  as  noise. 

In  real-world  systems,  false  matches  are  caused  by  a  combination 
of  the  two  mechanisms — spatial  pattern  repetition  and  other  system 
errors,  including  noise.  Generally  speaking,  mathematical  models  have 
been  developed  only  for  the  homogeneous  scene  case  with  the  basic 
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failure  mechanism  being  the  additive  noise  case.  The  nonhomogeneous 
scene  case,  even  for  the  simple  additive  noise  failure  mode,  is  much 
more  difficult  to  model,  and  being  very  scene  dependent  does  not  give 
a  great  deal  of  insight  into  the  interaction  between  cause  (error)  and 
effect  (change  in  system  performance) .  The  mathematical  modeling 
problem  becomes  even  more  difficult  when  one  attempts  to  model  the 
scene  structure,  potentially  introducing  nearly  repetitive  spatial 
patterns  into  the  process.  The  complicated  failure  mechanism,  the 
difficulty  in  modeling  the  scene  including  its  structure,  and  the  im¬ 
possibility  of  defining  a  "typical  scene"  forces  one  rather  rapidly 
into  a  simulation  mode  for  answers  to  the  effects  of  errors  on  system 
performance.  However,  that  is  not  to  say  that  some  of  the  mathematical 
models  are  not  useful  in  giving  one  an  initial  design  point  to  work 
from.  A  number  of  theoretical  analyses  have  provided  us  with  a  start¬ 
ing  point  for  the  simple  noise/homogeneous  scene  case,  but  the  more 
complicated  repetitive  spatial  pat tern /nonhomogeneous  case  has  not 
been  treated,  to  date,  mathematically . 


»  •/ 
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III.  METHODS  FOR  IMPROVING  SYSTEM  ACCURACY 


Section  II  laid  the  groundwork  for  understanding  the  accuracy  and 
false  match  issues.  In  this  section,  we  continue  that  discussion  and 
concentrate  on  methods  for  sharpening  the  correlation  peak.  We  focus 
on  the  utility  of  various  algorithms  for  improving  accuracy,  and  because 
accuracy  can  be  considered  a  separate  issue  from  the  acquisition  prob¬ 
lem,  we  are  not  concerned  with  the  problem  of  false  matches  that  is 
sometimes  aggravated  by  such  algorithms.  That  subject  is  treated  in 
a  subsequent  section  of  this  Note.  Finally,  this  Note  does  not  deal 
with  the  ultimate  system  accuracy  achievable,  which  involves  inter¬ 
polation  of  the  correlation  peak;  however,  one  might  expect,  based  on 
other  simulation  work,  that  accuracies  (CEP)  on  the  order  of  one  to 
two  pixels  can  be  expected. 

As  pointed  out  in  the  previous  section,  segmentation  of  the  scene 
into  homogeneous  regions  can  improve  system  accuracy.  This  can  be  done 
automatically  by  employing  either  a  pure  feature  matching  algorithm  or 
by  utilizing  a  feature  matching  correlation  algorithm.  The  other  prob¬ 
lem  is  to  improve  accuracy  in  the  presence  of  geometric  distortion. 

One  can  obviously  improve  the  local  accuracy  by  shrinking  the  size  of 
the  map  that  one  is  working  with;  however,  it  is  not  clear  (unless  the 
target  itself  is  a  small  distinctive  area  contained  within  the  sensed 
image)  that  the  overall  accuracy  can  be  improved  significantly  by 
locally  matching  parts  of  the  scene  due  to  the  mapping  error  resulting 
from  relating  multiple  points  of  match  to  the  target  area. 

To  illustrate  further  the  effects  on  accuracy  of  homogeneously 
segmenting  the  map,  we  took  a  20  x  20  element  map  from  mountain,  desert, 
and  suburban  areas.  Each  of  these  three  maps  had  more  than  one  homo¬ 
geneous  region.  Thirty-six  5*5  element  sensor  maps  were  correlated 
over  each  of  the  three  scenes,  and  a  number  of  scene  parameters,  in¬ 
cluding  Nj  and  the  scene  resolution,  N/N^  (the  indicator  of  system  ac¬ 
curacy),  were  computed  for  each  submap  in  the  region.  The  results  are 

*  ” 

DARPA's  Terminal  Homing  Program. 
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tabulated  in  Tables  2  through  4.  Also  shown  in  the  tables  are  the  mean 
and  standard  deviation  of  the  scene  resolution  for  each  area.  The 
results,  using  the  ordinary  or  standard  correlation  procedure  (normalized 
product  algorithm) ,  are  compared  later  for  the  same  maps  using  a  feature 
matching  correlation  algorithm  in  which  both  the  sensor  and  the  reference 
scenes  are  segmented  homogeneously. 

In  the  mountain  area  for  the  unprocessed  case,  the  scene  resolution 
varied  from  about  one  pixel  (which  is  an  accuracy  equal  to  the  sensor 
resolution)  to  almost  seven  pixels  with  an  average  value  of  three  pixels 
and  a  relatively  large  standard  deviation  of  1.7  pixels.  In  the  desert 
area,  the  scene  resolution  varied  from  about  two  pixels  to  about  five, 
with  a  mean  of  3.3  pixels  and  a  standard  deviation  of  0.8  pixel.  In 
the  suburbs,  the  scene  resolution  varied  from  one  to  three  pixels,  with 
a  mean  of  two  pixels  and  a  standard  deviation  of  0.8  pixel.  Thus,  even 
within  a  small  area  in  a  scene  there  can  be  significant  variations  in 
the  scene  resolution  because  of  the  variation  in  homogeneous  regions 
overlapped  by  the  sensor  maps  in  a  given  area.  This  variation  in  scene 
resolution  will  decrease  as  the  sensor  map  size  grows  (because  the  number 
of  homogeneous  regions  contained  in  a  large  map  generally  will  not  change 
drastically  in  a  given  region).  However,  even  though  the  standard  devi¬ 
ation  in  scene  resolution  will  decrease  with  increasing  map  size,  the 

* 

magnitude  of  the  scene  resolution  will  remain  high  in  nonhomogeneous 
scenes  compared  with  a  completely  homogeneous  map.  This  high  scene 
resolution  value  will  decrease  accuracy  in  the  correlation  process. 

By  segmenting  the  map,  the  effects  caused  by  the  interface  of  homo¬ 
geneous  boundaries  are  eliminated  and  the  scene  resolution  and  accuracy 
become  dependent  only  on  the  interpixel  correlation  in  the  segmented 
regions,  and  not  on  the  bias  level  difference  between  regions. 

Correlations  were  performed  using  the  feature  matching  correlation 
algorithm  for  each  of  the  submaps  in  each  of  the  three  areas.  The 
and  scene  resolution  parameters  for  this  case  are  also  shown  in  Tables 
2  through  4.  In  the  mountain  area,  the  scene  resolution  was  confined 
to  values  between  1  and  1.5  pixels,  with  an  average  value  of  1.3  pixels 
and  a  standard  deviation  of  0.2.  In  the  desert  area,  the  scene 

'ft 

Low  scene  resolution  N/Nj  «  1  (high  resolution)  high  accuracy; 
high  scene  resolution  N/Nj-  «  N  (low  resolution)  low  accuracy. 
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NO-NOISE  PARAMETERS  FOR  UNPROCESSED/HOMOGENEOUSLY  SEGMENTED  SCENES 


1, 


Unprocessed  scene  2.02  0.605 

With  homogeneous 

segmentation  1.2  6  0.22 
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resolution  varied  between  1.5  to  2.5  pixels,  with  a  mean  of  1.9  pixels 
and  a  standard  deviation  of  0.4  pixel.  In  the  suburban  area,  the  scene 
resolution  varied  between  1  and  2  pixels,  with  a  mean  of  1.2  pixels  and 
a  standard  deviation  of  0.2  pixel.  In  all  cases,  the  scene  resolution 
was  reduced  significantly  (a  factor  of  2  in  the  desert,  3  in  the  sub¬ 
urbs,  and  8  in  the  mountains)  and  the  variation  in  scene  resolution 
among  regions  (mountain,  desert,  and  suburbs)  was  less  pronounced. 

Thus,  system  accuracy  can  be  Improved  by  using  map  matching  algorithms 
which  emphasize  features  or  homogeneous  regions  in  the  scene. 

The  area  around  the  true  match  point  in  which  the  extremum  corre¬ 
lation  values  fall  is  indicative  of  the  accuracy  or  "acquisition  bas¬ 
ket."  To  test  the  utility  of  the  "scene  resolution"  as  an  estimator 
for  both  the  acquisition  basket  size  and  the  tightness  of  the  acquisi¬ 
tion  basket  when  a  feature  matching  correlation  algorithm  is  used,  a 
Monte  Carlo  simulation  was  run  for  a  number  of  different  scenes.  This 
Monte  Carlo  simulation  was  performed  for  both  the  ordinary  correlation 
algorithm  (normalized  product)  and  the  feature  matching  correlation 
algorithm  (both  the  reference  and  sensor  scenes  homogeneously  segmented) . 
This  simulation  consisted  of  taking  the  original  scene  and  adding  white 
Gaussian  noise  to  each  sensor  element,  correlating  the  images,  and 
locating  the  displacement  position  of  best  match.  This  process  was 
repeated  25  times  and  the  locations  of  the  correlation  matches  relative 
to  the  correct  match  location  are  recorded  in  Fig.  10.  Shown  in  this 
figure  are  representative  results  for  four  different  scene  resolutions 
using  the  ordinary  correlation  algorithm.  The  circled  position  indi¬ 
cates  the  location  of  the  true  match  and  the  number  in  each  displace¬ 
ment  position  represents  the  number  of  simulation  runs  for  which  that 
displacement  position  was  chosen  to  be  the  match  point.  As  illustrated 
in  this  figure,  the  larger  the  "scene  resolution"  value,  the  greater 
the  probability  that  a  number  of  the  match  points  will  occur  in  an  area 
adjacent  to  the  correct  match  point  (an  area  bounded  by  the  sensor 
resolution).  Thus,  the  scene  resolution  does  indicate  the  size  of  an 
acquisition  basket  around  the  true  match  point  where  the  correct  match 
points  are  likely  to  fall.  The  smaller  the  scene  resolution,  the  more 
likely  the  correct  match  point  will  occur  within  the  area  of  one  pixel 


Fig.  10  —  Accuracy  of  match  prediction  r*cant  rreoiution”) 
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(sensor  resolution).  As  the  "sensor  resolution"  is  decreased,  the 
correct  match  points  will  spread  out  over  a  larger  area  around  the 
correct  mate))  point  between  the  two  maps.  (The  area  size  of  this 
spread  will  roughly  equal  in  pixels  the  "scene  resolution.") 

To  test  the  segmenting  process,  several  5  x  5  subregions  were  se¬ 
lected  from  a  map  of  an  agricultural  area.  The  three  subregions  con¬ 
sist  of  Case  1,  a  sensor  map  which  is  completely  homogeneous;  Case  2, 
a  sensor  map  just  slightly  overlapping  a  second  homogeneous  area;  and 
Case  3,  a  sensor  map  straddling  the  boundary  between  two  homogeneous 

regions.  Monte  Carlo  simulations  were  run  for  these  regions  for  both 
* 

an  unprocessed  and  a  homogeneously  segmented  sensor  and  reference  map. 
The  match  point  locations  and  the  calculated  analytical  parameters  are 
shown  in  Fig.  11.  For  the  homogeneous  sensor  map,  Case  1,  the  results 
for  both  methods  of  correlation  are  almost  identical  with  the  variation, 
primarily  due  to  different  noise  samples.  As  the  degree  of  nonhomo¬ 
geneity  in  the  map  increases,  the  acquisition  basket  or  system  accuracy 
degrades  for  the  ordinary  correlation  case,  while  it  remains  essentially 
fixed  for  the  homogeneously  segmented  case,  as  measured  by  the  scene 
resolution  parameter. 

Geometric  errors  tend  to  spread  out  the  correlation  surface  and 
thus  degrade  the  overall  system  accuracy.  Honeywell,^  for  the  case  of 
a  two-dimensional  image,  has  developed  an  iterative  algorithm  known  as 
"address  modification"  for  removing  geometric  warping  between  images. 

The  basic  concept  is  that  geometric  distortions  between  the  reference 
and  sensor  scenes  can  be  represented  for  coplanar  scenes  by  an  eight- 
parameter  transformation.  Honeywell  has  developed  an  algorithm  which 
iteratively  converges  to  a  solution  for  these  geometric  parameters,  so 
that  one  can  remove  the  effects  of  geometric  distortion  from  the  image. 
This  algorithm  is  useful  in  improving  system  accuracy  once  the  acquisi¬ 
tion  problem  of  locating  the  general  area  of  match  between  the  two  images 
has  been  solved.  The  case  of  three-dimensional  images  (i.e.,  cultural 
features  such  as  buildings)  is  a  more  difficult  problem  to  dewarp  and 
requires  storing  the  entire  three-dimensional  image  of  the  target  area. 


Ordinary  correlation  algorithm. 

■f* 

J.  Merchant,  Address  Modification,  Image  Technology  Program, 
Vol.  I;  Overview  and  Theory,  Vol.  II,  Experimental  Results,  Honeywell 
Electro-Optics  Center,  Lexington,  MA,  September  1978. 
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Fig.  11  —  Comparison  of  no  scans  praproceating  with  homogeneous  scene  segmentation  Monts  Carlo  rasulas 
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In  summary,  the  following  points  can  be  made  concerning  the  accu¬ 
racy  obtainable  in  a  map  matching  system: 

General  Accuracy  Statements 

1.  Accuracy  can  be  qualitatively  estimated  by  examining  the  slope 
of  the  correlation  surface  surrounding  the  true  match  peak  and 
quantitatively  estimated  through  use  of  the  "scene  resolution" 
concept. 

2.  Accuracy  is  dependent  on  both  the  interpixel  correlation  and 
the  number  of  homogeneous  regions  in  the  scene. 

3.  Accuracy  is  most  severely  degraded  by  the  presence  of  large 
homogeneous  regions  without  structure  within  the  scene,  and 
by  the  introduction  into  the  system  of  geometric  distortions. 

Methods  for  Improving  Accuracy 

4.  Accuracy  can  be  improved  by  the  use  of  feature  matching  tech¬ 
niques  (pure  feature  matching,  feature  matching  correlation, 
or  hybrid  correlation)  which  homogeneously  segment  the  scene. 

5.  Accuracy  in  the  presence  of  geometric  distortions  can  be  im¬ 
proved  by  the  reduction  of  the  map  size  and  by  utilizing 
algorithms  that  dewarp  the  distortion  between  reference  scene 
and  sensed  image. 
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IV.  THE  ACQUISITION  PHASE  OF  MAP  MATCHING 


The  acquisition  phase  is  concerned  primarily  with  obtaining  a  match 
location  in  the  vicinity  of  the  correct  location  and  avoiding  at  all 
possible  costs  a  false  match;  hence,  the  performance  measure  of  interest 
is  the  probability  of  false  match,  Pp.  The  acquisition  system  design 
problem  is  illustrated  in  Fig.  12.  Basically  chere  are  a  number  of 
system  parameters  such  as  sensor  orientation,  resolution,  wavelength, 
and  the  flight  geometry  which  can  introduce  errors  into  the  system  and 
which  provide  the  basis  for  selecting  the  area  in  which  the  reference 
maps  are  located.  Various  parts  of  the  system  (preprocessing,  scene 
selection,  algorithm  choice,  and  system  verification)  are  then  designed 
to  accommodate  the  potential  problem  areas  (errors  and  scene  composition) 
The  system  design  approach  is  sequential  in  nature.  The  first  step  of 
the  design  procedure  is  the  preprocessing.  Global  errors,  primarily 
geometric  distortions,  must  be  accommodated  in  the  preprocessing  of  the 
acquisition  phase,  limiting  the  map  size  and  grouping  the  elements. 

Also,  the  intensity  level  normalization  plan  must  be  formulated  to  ac¬ 
commodate  anticipated  gain  changes  and  bias  shifts.  Preprocessing  sets 
the  stage  for  the  scene  selection  process  because  it  essentially  deter¬ 
mines  the  size  and  shape  of  the  sensor  image  or  subimages  (depending 
on  whether  or  not  the  scene  is  to  be  broken  into  segments). 

With  the  size  and  shape  of  the  sensor  map  governed  by  the  antici¬ 
pated  geometric  errors,  the  scene  selection  process  must  check  the  en¬ 
semble  of  all  possible  sensor  images  contained  within  the  candidate 
reference  areas  for  (1)  a  sufficient  number  of  independent  elements  (to 
ensure  that  sufficient  information  is  available  to  perform  matching) 
and  (2)  intrascene  spatial  redundancy  (to  avoid  as  much  as  possible  the 
checkerboard  problem  of  obtaining  many  similar  regions  within  a  scene) . 
Once  the  scene  selection  process  has  identified  a  reference  area  which 
has  a  sufficient  number  of  independent  elements  and  avoids  the  intra¬ 
scene  redundancy  problem,  the  next  step  is  the  algorithm  choice.  The 
choice  of  general  algorithm  is  dictated  by  the  magnitude  and  type  of 
errors  (regional,  local,  and  nonstructured)  that  are  present  in  the 


Problem  System  Parameters 


Fig.  12  —  Acquisition  system  design 
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system.  In  the  extreme  case,  if  local  errors  are  dominant  (e.g.,  addi¬ 
tive  noise) ,  then  ordinary  correlation-type  algorithms  are  called  for; 
if  regional  errors  (e.g.,  contrast  reversals)  are  dominant  in  the  sys¬ 
tem,  feature  matching  algorithms  are  the  obvious  choice.  A  different 
mix  of  these  errors  calls  for  a  different  choice  of  matching  algorithm. 
Finally,  the  sensor  and  guidance  systems  must  be  verified  by  simulation 
to  ensure  a  low  P„  value. 

r 

The  remainder  of  this  section  discusses  each  of  the  system  design 
steps  (preprocessing,  scene  selection,  algorithm  choice,  and  system 
verification)  separately. 

PREPROCESSING 

The  two  major  categories  of  preprocessing,  as  described  previously, 
are  intensity  level  modification  and  spatial  grouping  of  the  elements. 
Changes  in  gain  and  bias  levels  can  alter  the  sensor  signal  level.  If 
these  errors  are  present  in  the  system  then  the  data  must  be  modified 
(zero-meaned  for  bias  level  changes  and  normalized  to  unit  variance  for 
gain  changes)  to  accommodate  these  types  of  errors. 

Geometric-type  global  errors  must  be  accommodated  in  the  acquisition 
phase  by  reducing  the  number  of  pixels  contained  within  the  map  to  ac¬ 
commodate  the  maximum  amount  of  distortion  anticipated.  In  the  accuracy 
phase,  after  one  has  located  the  general  area  which  contains  the  correct 
match  location,  it  is  possible  to  use  other  techniques,  such  as  "address 

•I* 

modification,  to  remove  geometric  distortions.  Two  preprocessing 
methods  for  handling  the  geometric  distortion  problem  are  "spatial  win¬ 
dowing"  and  "subarea  correlation." 

The  concept  of  "spatial  windowing,"  introduced  by  Fred  Smith  of 
■J* 

SCI,  is  a  method  for  improving  both  the  sharpness  of  the  correlation 
peak  and  reducing  the  false  fix  probability.  This  concept  consists  of 
taking  several  different  size  sensor  maps,  all  spatially  centered.  The 
largest  sensor  map  or  window  about  the  center  point  will  have  the  flat¬ 
test  correlation  surface,  while  the  smallest  window  will  have  the 


Honeywell. 

^F.  W.  Smith  et  al.,  Optimal  Spatial  Filters ,  DARPA  Contract  DAAK 
40-77-C-0113,  System  Control,  Inc.,  Palo  Alto,  September  1978. 
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sharpest  peak;  however,  because  it  has  fewer  elements,  it  will  also 
have  many  sharp  false  peaks.  The  philosophy  behind  the  concept  is 
that  the  true  peak  should  occur  in  the  correlations  associated  with 
every  window  size;  while  false  peaks  may  appear  in  the  correlation 
surface  of  all  window  sizes,  they  will  not  show  up  at  the  same  displace¬ 
ment  position  in  the  correlation  surface.  Therefore,  by  screening 
correlation  peaks  as  a  function  of  displacement  position  for  all  win¬ 
dow  sizes,  one  can  possibly  eliminate  the  false  peaks  and  locate  the 
true  match  location.  One  possible  screening  procedure  would  involve 
first  setting  all  the  negative  correlation  values  in  the  correlation 
surface  equal  to  zero  and  then  multiplying  at  each  displacement  posi¬ 
tion  the  values  of  the  correlation  surface  for  each  window  size.  The 
process  of  setting  the  negative  correlation  values  equal  to  zero  would 
eliminate  the  possibility  of  having  the  product  of  several  large  nega¬ 
tive  correlation  values  coming  out  to  be  a  relatively  large  positive 
value.  The  true  correlation  peak  should  be  a  relatively  high  value, 
occurring  at  the  same  displacement  position  in  all  correlations;  false 
peaks,  not  occurring  at  the  same  location,  would  have  high  values  at 
one  displacement  position  for  one  sensor  map  size  offset  by  low  corre¬ 
lation  values  for  another  sensor  map  at  the  same  location.  Thus,  the 
multiplication  processing  of  several  correlation  surfaces  should  cause 
the  true  peak  to  stand  out  and  increase  the  accuracy  with  which  it  is 
located . 

"Subarea  correlation"  is  another  means  of  reducing  the  vulnerabil¬ 
ity  of  the  system  to  false  matches  when  geometric  errors  are  present. 

In  this  concept,  instead  of  matching  one  large  map  to  another  larger 
map,  a  number  of  smaller  maps  are  selected  from  the  reference  map  in 
such  a  manner  that  there  is  a  high  probability  that  several  of  them 
will  be  contained  within  the  sensor  map.  These  individual  subareas  are 
then  correlated  over  the  sensor  map  and  the  position  of  best  match  is 
determined  for  each  subarea.  Since  the  relative  location  of  the 

jUp 

G.  Gerson  et  al..  Image  Sensor  Measurements  Program:  Volume  1 , 
Multiple  Subarea  Bi-Level  Correlation  Scene  Matching  System ,  Contract 
F-30602-77-C-0049,  Hughes  Research  Laboratories,  Malibu,  California, 
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subareas  is  known  perfectly  in  the  reference  map,  this  information, 
combined  with  the  positional  alignment  of  the  subareas  in  the  sensor 
map,  will  enable  the  sensor  map  to  be  located  relative  to  the  reference 
map,  even  in  the  presence  of  geometric  distortions  in  the  scene.  One 
of  the  key  questions  in  this  concept  is  how  to  pick  the  subareas.  The 
subareas  should  be  chosen  such  that  (1)  they  are  geometrically  posi¬ 
tioned  so  that  multilateration  of  the  subareas  will  reveal  the  location 
of  a  reference  point  in  the  map  and  the  magnitude  of  any  geometric 
errors  in  the  scene,  and  (2)  each  subarea  can  be  uniquely  and  accurately 
located.  The  accuracy  criterion  involves  choosing  subareas  with  a  low 
"scene  resolution"  and  avoiding  subareas  that  have  contained  within 
them  the  shape  of  homogeneous  regions  repeated  elsewhere  in  the  scene. 

"Spatial  windowing"  and  "subarea  correlation"  are  primarily  de¬ 
signed  to  avoid  the  false  match  problem  and  identify  the  acquisition 
basket  in  which  the  true  peak  is  expected  to  be  located.  These  two 
techniques  are  also  useful  in  the  accuracy  phase  of  matching;  however, 
geometric  dewarping  techniques  theoretically  look  more  promising  in 
accommodating  distortion  in  two-dimensional  imagery. 

SCENE  SELECTION 

The  subject  of  criteria  for  scene  selection  is  almost  a  topic  of 
its  own.  One  must  choose  reference  areas  for  which,  for  the  ensemble 
of  possible  sensor  maps  contained  within  the  reference  area,  there  is 
(1)  sufficient  information  for  map  matching  and  (2)  a  minimal  amount  of 
interscene  spatial  redundancy  within  the  reference  map  boundary. 

Various  authors  have  suggested  a  number  of  measures  to  deal  with 
the  information  content  issue;  however,  no  procedures  have  been  pro¬ 
posed  to  deal  with  the  spatial  redundancy  question.  The  three  most 
relevant  measures  for  information  content  are  (1)  the  correlation 

length,  (2)  the  scene  resolution,  and  (3)  the  number  of  vertices  as- 

* 

sociated  with  features  in  the  scene  (proposed  by  Hall  ).  The 

*E.  Hall  and  R.  C.  Gonzalez,  Scene  Content  Analysis ,  Measurement , 
Refinement ,  and  Unification,  DARPA/SAMSO  Contract  F4701-77-C-0072, 
University  of  Tennessee,  December  1978. 


correlation  length  measure  is  used  by  many  authors  as  a  scene  content 
measure.  However,  as  pointed  out  previously,  correlation  length  is  a 
confusing  term  and  can  be  taken  to  mean  either  of  two  different  quan¬ 
tities.  In  the  case  of  a  completely  homogeneous  scene  it  is  meant  to 
be  the  degree  of  interpixel  correlation.  For  the  nonhomogeneous  case 
it  is  dominated  by  the  size  and  number  of  homogeneous  regions  in  the 
scene  and  not  by  the  interpixel  correlation.  The  other  major  problem 
with  this  measure  is  that  the  correlation  length  can  be  computed  for 
any  number  of  directions  around  the  correct  match  location  so  that 
there  is  an  additional  ambiguity  as  to  whether  one  is  dealing  with  the 
average  correlation  length,  longest,  shortest,  etc. 

A  measure  which  attempts  to  overcome  both  deficiencies — definition 
and  measurement — of  correlation  length  is  the  "scene  resolution"  con¬ 
cept.  Basically,  the  attempt  is  to  estimate  in  a  pseudo  sense  the 
number  of  independent  elements  in  the  scene.  By  dividing  this  number 
into  the  total  number  of  scene  elements,  one  obtains  an  estimate  of 

the  scene  resolution.  The  procedure  for  calculating  the  scene  resolu- 

,  * 

tion  is  explained  in  detail  in  an  earlier  Rand  report.  Briefly,  the 

basic  approach  in  calculating  the  scene  resolution  is  to  assume  a 
"statistically  equivalent  scene  model,"  i.e.,  that  the  scene  is  com¬ 
pletely  homogeneous  with  all  elements  independent.  Obviously  this 
assumption  is  not  true,  but  by  making  it  one  can  use  the  simple  rela¬ 
tionship  between  the  statistics  of  the  correlation  surface  and  the  num¬ 
ber  of  independent  scene  elements.  Thus,  by  simply  measuring  the  sta¬ 
tistics  of  the  correlation  surface,  one  can  work  backwards  to  estimate 
for  all  scenes  the  scene  resolution  (N/N^)  using  the  statistically 
equivalent  scene  model.  As  shown  in  the  previous  two  sections,  this 
parameter  does  perform  quite  well  in  predicting  (1)  the  size  of  the 
acquisition  basket  (the  region  around  the  correct  match  point  in  which 
correct  matches  are  likely  to  fall),  and  (2)  the  width  of  the  correla¬ 
tion  peak  (which  is  related  to  match  accuracy). 

*  •  •  • 

J.  A.  Ratkovic  et  al.,  Eetvmation  Techniques  and  Other  Work  on 

Image  Correlation ,  The  Rand  Corporation,  R-2211-AF,  September  1977. 
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One  potential  shortcoming  of  this  approach  is  that  it  is  a  corre¬ 
lation-based  technique  and  thus,  by  its  nature,  regions  or  features  in 
the  scene  are  weighted  by  the  number  of  sensor  elements  contained  with¬ 
in.  In  using  a  pure  feature  matching  technique  the  homogeneous  regions 
are  generally  weighted  equally  regardless  of  their  size.  Thus  Hall's 
measure  of  the  number  of  vertex  points  (N^)  of  homogeneous  regions  con¬ 
tained  within  the  scene  may  be  a  more  appropriate  measure  of  informa¬ 
tion  content  for  map  matching  when  pure  feature  matching  algorithms  are 
employed . 

As  pointed  out  in  Section  II,  the  relationship  between  the  occur¬ 
rence  of  false  matches  and  the  number  of  independent  scene  elements 
has  been  modeled  for  the  noise  failure  mechanism  but  not  for  the  spatial 
structure  failure  mechanism.  Without  a  mathematical  model  (which  would 
be  extremely  scene-dependent)  directly  relating  the  spatial  structure 
of  the  scene  to  system  performance,  one  can  only  speculate  what  measure 
might  be  indicative  of  a  spatial  structure  failure. 

Some  researchers  have  proposed  that  the  height  of  the  secondary 
peaks  in  the  correlation  surface  is  indicative  of  the  degree  of  con¬ 
gruence  between  the  sensor  image  and  some  other  area  in  the  reference 
scene  which  may  have  similar  characteristics.  This  quantity  may  not 
necessarily  be  a  good  measure  of  the  repetitive  spatial  structure  prob¬ 
lem  because  high  correlation  may  result  from  an  unknown  bias  or  gain 
change  since  standard  correlation  algorithms  are  global-type  operators. 
One  would  prefer  a  measure  more  related  to  the  underlying  structure  of 
the  scene  than  to  its  global  characteristics,  which  is  what  ordinary 
correlation  algorithms  work  on. 

The  hybrid  correlation  algorithm  provides  a  better  handle  on  this 
repetitive  scene  structure  problem.  This  algorithm  assumes  that  the 
sensor  scene  has  the  same  underlying  structure  as  the  reference  map  to 
which  it  is  being  compared.  This  algorithm  should  then  give  rise  to 
secondary  correlation  peaks  in  areas  where  the  repetitive  structure 
problem  is  likely  to  occur. 

Thus  it  appears  that  the  scene  resolution  concept  is  an  appropri¬ 
ate  measure  of  scene  content  for  correlation  algorithms,  whereas  the 
number  of  vertices  appears  to  be  appropriate  for  feature  matching 
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algorithms.  While  these  measures  (at  least  scene  resolution  and 
probably  the  measures)  can  be  related  to  the  probability  of  false 
fix  for  the  simple  noise  failure  case,  there  exists  no  such  measure  or 
model  for  the  spatial  repetition  problem.  Here  we  have  speculated  that 
the  height  of  secondary  correlation  peaks  (using  the  hybrid  algorithm) 
may  be  a  good  approach  in  measuring  the  degree  of  spatial  repetition 
in  the  reference  map. 

ALGORITHM  SELECTION  PROCESS 

The  choice  of  matching  algorithm  will  depend  on  the  nature  and 
magnitude  of  the  regional  and  local  errors.  Some  analysis  has  been 
performed  in  relating  nonstructured  errors  to  changes  in  system  per¬ 
formance.  In  general  the  algorithm  choice  is  not  strongly  dependent 
on  the  nature  of  nonstructured  errors.  Nonstructured  errors  are  best 
accommodated  in  the  mission  planning  phase  of  the  operation.  By  proper 
route  planning  obscuration  and  masking  errors  may  be  avoided,  and  by 
timing  and  weather  planning  it  may  be  possible  to  reduce  the  diurnal 
and  weather  effects  which  can  cause  nonstructured  errors.  Thus  non¬ 
structured  errors  can  be  reduced  by  careful  mission  planning.  Gener¬ 
ally,  any  residual  nonstructured  errors  cannot  be  adequately  modeled, 
and  thus  one  can  only  hope  that  they  do  not  seriously  degrade  system 
performance. 

The  algorithm  choice,  then,  in  the  extreme  case  of  local  errors 
only  tends  toward  ordinary  correlation,  whereas  in  the  other  extreme 
(regional  errors  only)  the  algorithm  tends  toward  pure  feature  matching. 
Because  one  is  generally  never  confronted  by  an  either-or  situation 
(except  in  the  case  of  terrain  contour  mapping,  where  there  are  primar¬ 
ily  local  errors),  it  is  necessary  to  weigh  the  relative  magnitude  of 
local  and  regional  errors  present  when  deciding  upon  the  choice  of  algo¬ 
rithm. 

Let  us  first  consider  the  differences  between  the  various  cate¬ 
gories  of  correlation  algorithms  when  only  local  errors  (additive  noise) 
are  present.  To  examine  the  effect,  we  took  several  10  *  10  element 
sensor  maps  from  the  center  of  20  x  20  reference  scenes  in  various 
parts  of  an  Earth  Resources  Satellite  map.  To  these  sensor  scenes,  we 
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ad  ded  white  Gaussian-distributed  noise  such  that  the  S/N  ratio  was  0.5. 
The  simulation  consisted  of  creating  25  different  noisy  sensor  images 
and  matching  the  reference  and  sensed  imagery  for  different  categories 
of  algorithms  (feature  matching  correlation,  hybrid,  and  ordinary  cor¬ 
relation)  and  types  of  algorithms  (product,  MAD,  and  difference  squared). 
Table  5  shows  the  number  of  independent  elements  estimated  to  be  con¬ 
tained  within  the  scene  and  the  percent  of  successful  matches  f°r 

each  category  and  type  of  algorithm. 

The  homogeneous  regions  within  the  reference  map  boundary  were 
defined  manually.  The  homogeneous  regions  or  features  in  the  sensor 
image  were  also  defined  manually  for  the  feature  matching  correlation 
algorithm.  In  the  real  world  these  regions  must  be  extracted  automati¬ 
cally  so  that  the  results  for  the  feature  matching  correlation  algorithm 
are,  in  a  sense,  an  optimum  case.  In  the  real  world,  homogeneous 
regions  are  generally  extracted  through  the  use  of  edge  operators, 
which  do  not  perform  well  in  the  presence  of  local  errors.  Simulation 
results  achieved  for  real-world  scenes  using  pure  feature  matching  ap¬ 
proaches  generally  indicate  that  results  closer  to  those  achieved  by 

the  hybrid  algorithm  are  obtainable  when  automated  edge  finding  feature 

* 

extraction  techniques  are  used. 

In  examining  the  simulation  results,  the  "perfect"  (homogeneous 
regions  extracted  from  both  the  map  and  the  sensor  image  without  error) 
feature  matching  correlation  algorithm  is  the  best  performer,  followed 
closely  by  the  ordinary  correlation  algorithm.  The  hybrid  class  of  al¬ 
gorithm  does  not  do  nearly  as  well  as  the  other  two  classes  of  algorithms 
when  the  product  algorithm  is  used.  Among  the  types  of  algorithms 

used,  the  difference  squared  was  definitely  the  best  performer,  pri¬ 
marily  because  there  were  no  gain  changes  in  the  data  and  the  product 
algorithm  was  normalized  in  the  preprocessing  for  a  gain  change 

*R.  L.  Kin,  Pattern  Matcher  Development  Study,  DARPA  Contract 
DAAK  40-77-C-0017,  Rockwell  International,  Missile  Systems  Division, 
Anaheim,  California,  June  1978. 

This  is  because  the  hybrid  algorithm  is  designed  to  accommodate 
both  regional  and  local  errors.  In  the  absence  of  regional  errors,  it 
would  be  expected  that  ordinary  correlation  techniques  (designed  only 
for  local  errors)  would  work  best. 
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processing  refers  to  whether  or  not  the  sensor  and/or  the  reference  maps  were  segmented  into  homogeneous 
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error.  Had  a  gain  change  occurred ,  the  product  (normalized)  algorithm 
would  have  been  the  superior  performer. 

The  simulation  results  also  indicated  that,  with  the  exception  of 
Region  12,  most  of  the  scenes  correlated  quite  well  when  considering 
the  magnitude  of  the  noise  error.  Thus,  for  most  of  these  cases,  the 
few  false  fixes  that  did  occur  were  due  primarily  to  a  noise  failure 
mechanism.  The  unusual  performance  of  the  agricultural  scene  (Region 
12)  might  lead  one  to  suspect  an  intrascene  spatial  redundancy  failure 
mode,  which  was  indeed  the  case. 

Figure  13  shows  the  pixel-by-pixel  breakdown  of  the  homogeneous 
segmentation  process  (indicated  by  region  numbers)  and  the  location  of 
the  25  match  points  for  the  simulation  runs  where  only  the  reference 
map  was  broken  into  homogeneous  regions.  Two  different  map  comparison 
positions  are  outlined  in  the  figure — the  correct  match  sensor  location, 
taken  from  the  center  of  the  scene,  and  a  map  comparison  position  in 
the  upper  left-hand  portion  of  the  scene.  The  relative  match  location 
in  the  simulation  runs  is  indicated  in  the  figure.  Examination  of  the 
two  map  displacement  positions  reveals  an  almost  identical  spatial 
pattern  of  the  homogeneous  regions  for  both  displacement  positions. 

This  repetition  of  the  spatial  pattern  led  to  a  large  number  of  false 
fixes  for  the  difference  squared  algorithm  at  the  repeated  position. 

For  the  product  algorithm,  no  false  fixes  occurred  at  the  pattern 
repeat  position;  however,  a  large  number  did  occur  at  nearby  locations. 

The  Appendix  contains  intensity  level  and  region  maps  for  these 
scenes,  as  well  as  the  autocorrelation  surfaces  using  all  three  al¬ 
gorithms.  The  autocorrelation  surface  using  ordinary  correlation 
shows  for  Region  12  a  high  correlation  coefficient  in  the  displacement 
position  where  false  matches  did  occur.  The  hybrid  correlation  surface 
shows  a  secondary  peak  at  this  location;  while  the  correlation  value 
was  not  as  high  (0.6  versus  0.8  for  the  ordinary  correlation),  it  is 
definitely  easier  to  locate  this  intrascene  redundancy  problem  from 
the  hybrid  algorithm  as  opposed  to  the  ordinary  correlation  algorithm. 
The  feature  matching  correlation  algorithm  did  not  yield  (as  indicated 
by  the  correlation  surface)  a  great  deal  of  information  concerning  the 
magnitude  and  location  of  the  intrascene  redundancy  problem.  In 
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Region  2,  the  hybrid  correlation  algorithm  also  revealed  a  secondary 
peak  in  the  general  vicinity  where  a  false  match  did  occur  using  all 
three  ordinary  correlation  algorithms  (MAD,  product,  and  difference 
squared).  Thus  it  appears,  based  on  this  limited  sampling  of  data, 
that  the  hybrid  correlation  algorithm  can  play  a  significant  role  in 
determining  the  magnitude  and  location  of  the  spatial  scene  redundancy 
problem  during  the  scene  selection  process. 

To  further  test  the  suitability  of  an  algorithm  to  tolerate  local 
error,  another  simulation  identical  to  the  first  was  run  using  a  small 
sensor  map  (5  *  5).  The  smaller  sensor  map  with  fewer  elements  makes 
the  system  more  susceptible  to  both  noise  and  spatial  structure  failure 
modes.  For  this  particular  simulation,  only  the  ordinary  correlation 
algorithm  (scene  not  homogenized)  and  the  feature  matching  correlation 
algorithm  (scene  homogenized)  were  used.  With  the  smaller  map,  the 
ordinary  correlation  algorithm  generally  does  slightly  better  (in 
terms  of  choosing  the  correct  match  point  more  frequently) ,  as  is  in¬ 
dicated  in  Table  6,  than  the  "perfect"  feature  matching  correlation 
algorithm.  The  major  exception  is  a  region  where  there  is  a  large 
scene  redundancy  problem  (desert  region  5,  subregion  (6,1)). 

Thus,  if  one  could  screen  out  the  scenes  in  the  selection  process 
where  spatial  redundancy  is  likely  to  be  a  problem,  the  ordinary  cor¬ 
relation  algorithm  definitely  does  as  well  as  the  "optimum"  or  "perfect" 
feature  matcher  in  the  presence  of  large  local  errors.  When  one  con¬ 
siders  the  degradation  in  performance  encountered  in  going  from  the 
optimal  to  the  real-world  feature  matching  algorithms,  one  is  likely 
to  expect  performance  significantly  below  that  of  the  ordinary  correla¬ 
tion  algorithms  and  close  to  that  obtained  using  the  hybrid  class  of 
algorithms. 

To  determine  the  change  in  system  performance  due  to  regional  er¬ 
rors  interacting  with  the  three  different  categories  and  types  of  al¬ 
gorithms  described  previously,  we  ran  an  experiment  to  test  the  effects 
of  such  errors.  In  an  attempt  to  place  regional  errors  into  the  cor¬ 
relation  process,  we  decided  to  see  the  effect  of  changing  the  mean 
values  of  the  "intensity"  levels  in  the  homogeneous  regions  of  the 
scene.  For  this  experiment  a  sensor  map  (20  *  20)  was  chosen  with  a 
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larger  number  of  homogeneous  regions  (mountain  area,  region  4)  and  the 
mean  level  of  each  homogeneous  region  was  changed  by  a  random  amount. 
The  magnitude  of  the  level  change  was  drawn  from  a  zero-mean  Gaussian 
distribution  with  three  different  standard  deviations  chosen  to  be  25, 
50,  and  100  percent  of  the  dynamic  range  of  intensity  values  in  the 
scene.  Three  different  algorithms  (the  normalized  product,  the  differ¬ 
ence  squared  with  the  mean  intensity  value  subtracted  out,  and  the 
difference  squared  algorithm)  artd  three  different  processing  schemes 
(both  sensor  and  reference  maps  homogeneously  segmented,  only  the 
reference  map  segmented,  and  no  segmentation)  were  utilized.  Addition¬ 
ally,  a  small  amount  of  noise  was  added  to  each  pixel  in  the  scene. 

The  results  in  Table  7  give  the  percent  of  successful  correlations  (out 
of  25) ,  PCTU,  for  each  run  using  the  different  algorithm  categories 
and  types.  Since  we  are  using  the  "perfect"  feature  matching  correla¬ 
tion  algorithm,  we  would  not  expect  any  change  in  performance  with 
change  in  level,  and  the  results  so  indicate.  On  the  other  hand,  there 
is  a  definite  degradation  in  P_T„  for  the  ordinary  correlation  cases 
for  all  types  of  algorithms  with  increasing  changes  among  homogeneous 
levels  in  the  scene.  The  hybrid  algorithms,  while  generally  having 
performance  somewhat  below  that  of  the  "perfect"  feature  matching  al¬ 
gorithms,  essentially  do  not  degrade  with  increasing  regional  error. 

For  the  unprocessed  case,  the  difference  squared  and  normalized 
product  started  off  at  the  25  percent  intensity  level  change  performing 
about  equally  well;  however,  as  the  magnitude  of  the  level  change 
reached  50  percent  and  beyond,  the  normalized  product  algorithm  out¬ 
performed  both  versions  of  the  difference  squared  algorithm.  Thus,  if 
there  are  mean  intensity  level  changes  in  the  scene  on  the  order  of  50 
percent  of  the  intensity  difference  between  regions  of  the  scene,  and 
if  no  homogeneous  processing  is  performed,  the  normalized  product  al¬ 
gorithm  should  be  utilized.  If  either  the  reference  and/or  reference 
and  sensor  maps  are  homogeneously  segmented,  then  the  difference 
squared  algorithm  (zero-meaned)  should  be  used,  assuming  that  gain 
changes  are  not  anticipated. 
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Table  7 

SIMULATION  RESULTS  WITH  LEVEL  CHANGES  BETWEEN  HOMOGENEOUS  REGIONS 

Mountain  Area 
20  x  20  Sensor  Map 
40  x  4Q  Reference  Map 


Magnitude  of  Level  Change*5 

]  25  Percent  ] 

50  Percent 

100  Percent 

Process3 

Algorithm 

wEsm 

p 

SIM 

p 

SIM 

0 

Normalized  Product 

0.92 

0.88 

0.52 

0 

Difference  Squared 
(Zero-meaned) 

0.88 

0.68 

0.48 

0 

Difference  Squared 

0.88 

0.56 

0.40 

1 

Normalized  Product 

0.72 

0.72 

0.68 

1 

Difference  Squared 
(Zero-Meaned) 

0.96 

1.0 

1.0 

1 

Difference  Squared 

0.92 

0.64 

0.40 

2 

Normalized  Product 

1.0 

1.0 

1.0 

2 

Difference  Squared 
(Zero-Meaned) 

1.0 

1.0  j 

_ J 

1.0 

Process  refers  here  to  the  degree  of  homogeneous  segmentation, 
with  a  zero  indicating  ordinary  correlation,  a  one  indicating  that 
only  the  reference  map  was  segmented,  and  a  two  indicating  that 
both  maps  were  segmented. 

^In  the  simulation  runs  a  small  amount  of  additive  noise  was 
added  to  each  pixel  and  a  constant  random  value  was  added  to  each 
region.  The  magnitude  of  the  constant  value  was  determined  by  call¬ 
ing  a  random  number.  The  distribution  associated  with  the  random 
number  had  a  standard  deviation  equivalent  to  25,  50,  and  100  per¬ 
cent  of  the  range  of  intensity  values  in  the  scene. 

c 

PSIM  indicates  the  percent  of  correlations  out  of  25  that  fall 
within  an  area  the  size  of  the  scene  resolution  about  the  correct 
match  location. 
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Let  us  examine  the  magnitude  of  the  regional  errors  associated 

* 

with  various  sensor  types.  Table  8  gives  a  rough  guideline  of  the 
magnitude  of  the  regional  errors  one  would  anticipate  for  each  of  the 
sensor  types.  These  numbers  represent  general  guidelines  and  there 
a  number  of  caveats.  The  percentage  error  for  the  passive  visible/ 
near  IR  sensors  assumes  that  there  is  some  means  of  calibrating  the 
system  onboard  in  real  time  to  skylight  illumination.  The  middle  IR 
and  far  IR  wavelengths  do  not  operate  at  dawn  or  dusk  when  thermal 
crossovers  are  likely  to  occur,  and  it  is  assumed  that  one  has  a  refer¬ 
ence  map  that  differentiates  between  night  and  day  operations;  other¬ 
wise  the  regional  variations  can  be  as  high  as  a  few  hundred  percent. 
The  passive  microwave  is  very  material-dependent ,  with  high  emissivity 
material  (such  as  grass  and  soil)  behaving  similarly  to  the  middle  and 
far  IR  wavelengths;  the  low  to  moderate  emissivity  materials  can  be 
fairly  stable  in  signature,  with  the  possible  exception  imposed  by 
cloud  cover.  Active  microwave  signatures  can  vary  on  the  same  order 
as  the  middle  and  far  IR  signatures.  Range  signature  errors  such  as 
..nose  associated  with  TERCOM  are  generally  quite  small. 

Based  on  this  first-order  signature  analysis,  the  magnitude  of 
the  regional  errors  is  small  enough  for  the  active/near  IR  wavelengths 
and  for  the  range  sensors  so  that  ordinary  correlation  techniques  can 
perform  the  matching  task.  It  may  also  be  possible  to  use  ordinary 
correlation  for  a  passive  visible/near  IR  system.  For  all  other  wave¬ 
lengths  it  appears  that  one  must  move  away  from  the  ordinary  correla¬ 
tion  approach  and  incorporate  some  form  of  feature  matching  or  hybrid 
correlation  algorithm  into  the  matching  process  to  compensate  for 
regional  errors.  It  may,  however,  still  be  possible  to  use  ordinary 
correlation  algorithms  over  a  br^aaer  range  of  sensor  wavelengths  by 
carefully  screening  the  reference  area  with  respect  to  such  parameters 
as  emissivity,  thermal  inertia,  reflectivity,  etc. 

To  summarize,  ordinary  correlation  algorithms  look  attractive  for 
accommodating  local  errors  as  long  as  the  scene  spatial  redundancy 


Prepared  by  Lloyd  Mundie  of  Rand  and  Ed  Conrow  of  the  Aerospace 
Corporation. 


REGIONAL  ERRORS  VERSUS  SENSOR  TYPE 


Sensor  Type 

Percent  Regional  Error4 

Active  visible /near  IR 

10  to  20 

Passive  visible/near  IR^ 

20  to  40 

Middle  IR/ far  IR 

Greater  than  50 

Passive  microwave 

30  to  50 

Active  microwave 

Greater  than  50 

Range  sensors 

5 

aThe  average  difference  between  the  actual 
mean  intensity  level  difference  among  regions 
and  the  predicted  intensity  level  difference 
among  regions  divided  by  the  actual  intensity 
range  spread  among  regions. 

^ Assumes  sensor  calibrated  by  skylight  illu¬ 
mination. 

problem  can  be  avoided  in  the  scene  selection  process.  Ordinary  cor¬ 
relation  algorithms  degrade  significantly  in  the  presence  of  regional 
errors,  especially  when  the  magnitude  of  the  mean  intensity  level 
change  is  of  the  order  of  50  percent  or  more  of  the  difference  in 
intensity  levels  between  regions.  Feature  matching  correlation  al¬ 
gorithms  look  attractive  for  both  regional  and  local  errors  if  the 
homogeneous  boundaries  can  be  extracted  from  the  sensed  image. 

In  this  Note  we  have  been  working  with  perfect  feature  matching 
algorithms  (i.e.,  we  have  been  able  to  correctly  identify  the  homo¬ 
geneous  regions  in  the  sensor  map) .  In  the  real  world  this  is  gener¬ 
ally  difficult  because  of  local  (e.g.,  noise)  and  nonstructural  (e.g., 
shadows)  errors.  In  addition,  there  is  a  considerable  amount  of  com¬ 
puter  software  and  hardware  required  to  perform  this  feature  extraction 
process  in  near  real  time. 

Because  it  does  accommodate  regional  errors,  the  hybrid  algorithm 
offers  three  major  advantages.  First,  it  is  not  subject  to  the  prob¬ 
lems  of  extracting  features  from  the  sensed  image  which  pure  feature 
matching  algorithms  are  associated  with.  Second,  it  can  accommodate 
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regional  errors  which  can  cause  false  matches  in  ordinary  correlation. 
Third,  this  algorithm  does  not  require  the  extensive  computer  hardware 
and  software  of  the  feature  matching  algorithms.  The  only  major  draw¬ 
back  of  the  algorithm  is  that  it  appears  to  have  difficulty  operating 
in  a  very  low  S/N  ratio  (<  1)  environment.  In  general  this  drawback  is 
not  serious  if  the  entire  system  is  designed  properly.  It  appears  that, 
due  to  the  nature  of  the  feature  extraction  problem  using  edge  methods, 
alternative  statistical  approaches  need  to  be  developed  and  tested. 


V.  SUMMARY  AND  CONCLUSIONS 

This  Note  has  described  the  image  matching  process  as  a  two-phase 
process:  the  first  phase  is  acquisition  of  the  correct  match  area  and 
the  second  is  accurate  location  of  the  match  point.  The  major  reason 
for  the  failure  of  the  system  to  acquire  is  due  to  a  combination  of 
noise  plus  interscene  redundancy  (e.g.,  checkerboard);  the  latter 
problem  is  extremely  difficult  to  model.  Accuracy  depends  on  two  com¬ 
ponents  of  the  scene  structure — the  size  and  magnitude  of  homogeneous 
regions  in  the  scene  and  the  interpixel  correlation  (expressed  in  terms 
of  independent  scene  elements) — and  the  amount  of  geometric  distortion 
present. 

Accuracy  can  be  improved  by  using  a  hybrid  or  feature  matching  al¬ 
gorithm  which  segments  the  scene  into  homogeneous  regions.  This  seg¬ 
mentation  significantly  sharpens  the  correlation.  The  residual  spread 
in  the  correlation  peak  can  be  attributed  to  interpixel  correlation. 

For  coplanar  scenes,  geometric  distortions  between  the  reference  and 
sensor  scenes  can  be  represented  by  an  eight-parameter  transformation. 
Honeywell  has  developed  an  algorithm  which  iteratively  converges  to 
estimates  of  these  geometric  parameters,  so  that  one  can  remove  the 
effect  of  geometric  distortion  from  the  image.  Thus,  by  selecting 
algorithms  of  the  feature  matching  type,  and  by  incorporating  an  al¬ 
gorithm  for  removing  distortion,  it  is  possible  to  improve  match  ac¬ 
curacy. 

The  acquisition  problem,  described  in  Fig.  12,  consists  of  deter¬ 
mining  the  preprocessing  requirements,  developing  scene  selection 
criteria,  choosing  an  algorithm,  and  verifying  the  system  via  a  simula¬ 
tion.  As  indicated  in  this  figure,  the  first  problem  is  global  errors, 
which  are  generally  accommodated  by  either  normalizing  the  intensity 
level  or  by  spatially  grouping  the  scene  elements  to  reduce  the  sus¬ 
ceptibility  of  the  matching  process  to  geometric  distortion. 

The  scene  selection  process  requires  that  two  criteria  be  met. 

The  first  is  that  a  sufficient  amount  of  independent  information  be 
contained  in  the  map.  A  number  of  methods  have  been  proposed  to 
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measure  the  independent  information  contained  within  the  scene.  The 
correlation  length  appears  to  be  a  poor  measure  because  of  the  ambi¬ 
guity  associated  with  the  term.  The  number  of  "independent  scene 
elements"  appears  to  be  a  good  measure  for  correlation  processes, 
while  the  "number  of  vertices"  appears  appropriate  for  pure  feature 
matching  processes.  The  second  scene  selection  process  of  importance 
is  the  avoidance  of  interscene  redundancy  (e.g.,  checkerboard  patterns). 
The  height  of  secondary  correlation  peaks  using  ordinary  correlation 
does  not  appear  to  be  as  good  a  measure  of  scene  redundancy  as  the 
height  of  secondary  peaks  using  the  hybrid  algorithm.  This  hybrid 
class  of  algorithm  assumes  that  at  each  displacement  position  the 
sensor  image  is  segmented  into  homogeneous  regions  in  an  identical 
manner  to  the  portion  of  the  reference  map  against  which  it  is  being 
compared.  Thus,  this  class  of  algorithm  emphasizes  the  spatial  struc¬ 
ture  of  the  scene  and  the  few  simulation  results  acquired  to  date  in¬ 
dicate  that  secondary  peaks  on  the  autocorrelation  surface  associated 
with  the  hybrid  algorithm  are  places  where  false  matches  are  likely 
due  to  interscene  redundancy. 

Finally,  in  the  acquisition  process,  an  algorithm  must  be  chosen 
from  the  generic  class  of  ordinary  correlation,  hybrid  correlation, 
feature  matching  correlation,  and  feature  matching  such  that  it  can 
accommodate  the  numbers  of  regional,  local,  and  nonstructured  errors 
that  are  anticipated.  If  only  local  errors  are  anticipated  (e.g.,  in 
the  TERCOM  navigation  system),  ordinary  correlation  algorithms  are 
appropriate,  whereas  if  regional  errors  dominate,  a  feature  matching 
or  hybrid  algorithm  is  demanded.  Most  real-world  scenes  have  both 
regional  and  local  errors  superimposed.  If  the  magnitude  of  the  varia¬ 
tion  in  the  mean  intensity  levels  between  homogeneous  regions  in  the 
area  (that  can  be  accounted  for  in  the  signature  prediction)  exceeds 
in  value  50  percent  of  the  intensity  level  difference  between  regions, 
then  one  is  forced  to  use  a  feature  matching  algorithm,  with  the  hybrid 
algorithm  looking  like  an  attractive  alternative  to  avoid  the  near- 
real-time  feature  extraction  process  in  the  sensed  image  while  at  the 
same  time  being  able  to  deal  with  regional  errors. 
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REGION  MAPS  AND  CORRELATION  SURFACES  FOR  ORDINARY  CORRELATION.  HYBRID 
CORRELATION,  AND  FEATURE  MATCHING  CORRELATION  ALGORITHMS 


A-l. 
A-2. 
A- 3. 
A-4. 
A-5. 
A- 6 . 
A-7. 
A-8 . 
A-9. 
A-10. 
A- 11. 
A-12. 
A-13. 
A- 14. 
A- 15. 
A- 16. 
A- 17. 
A- 18. 
A- 19. 
A-20. 


Region  2  Intensity  Level  and  Region  Map 

Region  2  Correlation  Surface — Ordinary  Correlation 

Region  2  Correlation  Surface — Hybrid  Algorithm 

Region  2  Correlation  Surface — Feature  Matching  Algorithm 

Region  17  Intensity  Level  and  Region  Map 

Region  17  Correlation  Surface — Ordinary  Correlation 

Region  17  Correlation  Surface — Hybrid  Algorithm 

Region  17  Correlation  Surface — Feature  Matching  Algorithm 

Region  10  Intensity  Level  and  Region  Map 

Region  10  Correlation  Surface — Ordinary  Correlation 

Region  10  Correlation  Surface — Hybrid  Algorithm 

Region  10  Correlation  Surface — Feature  Matching  Algorithm 

Region  6  Intensity  Level  and  Region  Map 

Region  6  Correlation  Surface — Ordinary  Correlation 

Region  6  Correlation  Surface — Hybrid  Algorithm 

Region  6  Correlation  Surface — Feature  Matching  Algorithm 

Region  12  Intensity  Level  and  Region  Map 

Region  12  Correlation  Surface — Ordinary  Correlation 

Region  12  Correlation  Surface — Hybrid  Algorithm 

Region  12  Correlation  Surface — Feature  Matching  Algorithm 
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Fig.  A-t  —  Region  2  intensity  level  and  region  map 
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Fig.  A-2  * —  Region  2  correlation  surface— ordinary 


Fig.  A-3  —  Region  2  correlation  surface— hybrid  correlation 
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Fig.  A- 7  —  Region  17  correlation  surface -hybrid  algorithm 
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Fig.  A-13  —  Region  6  intensity  level  and  region  map 
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Fig.  A- 16  —  Region  6  correlation  surface— feature  matching  algorithm 
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Fig.  A- 17  —  Region  12  intensity  level  and  region  map 
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